Fluctuation Theorems on Nishimori Line 



Masayuki Ohzeki 
Department of Systems Science, Kyoto University, 
Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, Japan 
(Dated: October 31, 2012) 

The distribution of the performed work for spin glasses with gauge symmetry is considered. With 
the aid of the gauge symmetry, which leads to the exact /rigorous results in spin glasses, we find 
a fascinating relation of the performed work as the fluctuation theorem. The integral form of the 
resultant relation reproduces the Jarzynski-type equation for spin glasses we have obtained. We show 
that similar relations can be established not only for the distribution of the performed work but also 
that of the free energy of spin glasses with gauge symmetry, which provides another interpretation 
of the phase transition in spin glasses. 



I. INTRODUCTION 



The fluctuation theorem makes current activities to 
understand the nonequilibrium behavior [1-8]. The the- 
orem consists of a relation between distribution func- 
tions in different conditions. The first discovery was 
in long time observation for an entropy production rate 
0, Q • The rigorous derivation of the fluctuation theorem 
was given by Gallavotti and Cohen for the thermostated 
deterministic steady-state ensembles [U, [|| and for the 
stochastic dynamics The fluctuation theorem for 

the nonequilibrium behavior as well as for the symmetry 
broken states in equilibrium have been discovered |j[ . In 
the present study, we focus on the fluctuation theorem 
for the performed work distributions [1|. This type of 
the fluctuation theorem yields a fascinating relation for 
the expectation of the exp onentiated work known as the 
Jarzynski equality [Tol |ll| . The Jarzynski equality makes 
a relationship between the equilibrium free energy differ- 
ences associated with the initial and final conditions and 
the performed work during a nonequilibrium process. 

Recently the author and co-worker investigated the 
Jarzynski equality for spin glasses with competing inter- 
actions between adjacent spins [12, EH- Spin glasses often 
exhibit extremely long-time relaxation toward equilib- 
rium. The long equilibration time hampers observations 
of the equilibrium state of spin glasses. However the pre- 
vious study pointed out the possibility to investigate the 
equilibrium property from the observations in a different 
path through the nonequilibrium behavior. Then we use 
the property of the Jarzynski equality by evaluating the 
average of the exponentiated performed work during the 
nonequilibrium process. In other words, nonequilibrium 
behavior would not be nuisance but benefit to investigate 
the equilibrium behavior in spin glasses. In the present 
study, we address the distribution of the performed work 
in such beneficial nonequilibrium behavior with long-time 
equilibration in spin glasses. Such fundamental research 
has never been performed as far as the best of our knowl- 
edge. We revisit the analysis of the nonequilibrium be- 
havior in spin glasses in terms of the distribution function 
of the performed work by evaluation of the rate function. 
As a result, we obtain a relation in a similar form to 



the conventional fluctuation theorem for the performed 
work for spin glasses. The similar analyses reveal that 
the gauge symmetry, which leads to simple expressions 
of several quantities for spin glasses [HI, [Hj], is closely 
related to the existence of the fluctuation-theorem type 
relation. 

The paper is organized as follows. The second section 
gives a brief introduction of several notations and tools in 
spin glasses. In the third section, we review the previous 
study on the performed work by use of the Jarzynski 
equality. In the present study we give the fluctuation 
theorem in spin glasses by employing several techniques 
developed in spin glasses. We show the detailed analysis 
in §IV. The analyses with the aid of the specialized tool 
to spin glasses can yield other types of the fluctuation 
theorem rather than for the performed work. We show 
the fluctuation theorems for the free energy differences 
and free energy itself in the following sections. In the 
last section, we conclude our present work. 



II. SPIN GLASS AND GAUGE SYMMETRY 

We deal with the random-bond Ising model, whose 
Hamiltonian is defined as 



ff(S|{r«}) 



(ij) 



(1) 



where Si is the Ising spin taking values ±1, J denotes 
strength of the coupling, and is the sign of the cou- 
pling. We use the notation S = (Si, S*2, • • • , Sjv) for the 
spin configuration of total N spins for convenience. We 
set J = 1 without loss of generality. The summation 
is taken over all bonds, whereas one may suppose usual 
nearest neighboring bonds on <i-dimensional hyper-cubic 
lattice. We make no restrictions on the type or the di- 
mension of the lattice in the present study. The distri- 
bution function of quenched randomness is specified as 

P{n 3 ) = P 6( nj - 1) + (1 - p)S( nj + 1) = 2 l o l^ p , (2) 

where j3 p is defined as e~ 2 ^ p = (1 —p)/p. The following 
analyses can readily be applied to other types of interac- 
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tion and its distribution functions as long as they satisfy 
a certain type of gauge symmetry [lil . fig . 

We use the gauge transformation, which enables us to 
perform the exact/rigorous analyses in spin glasses espe- 
cially on a special subspace j3 = f3 p known as the Nishi- 
mori line [IH Il5j . The gauge transformation is defined 
as 

Tj — > TijOiOj Si — > CTiSi, (3) 

where cr, is the gauge variable taking ±1. The Hamilto- 
nian fl} is gauge invariant, while the distribution func- 
tion ([2]) changes as P{Tj) oc exp(/3 p r ij (T i (Tj). If we take 
the summation over all combinations of {<?i}, the product 
of the distribution function for {ry} over all bonds can 
be reduced to the partition function of the random-bond 
Ising model Z(f3 p ; {r^}) = J2 a ex P(/3 P Uj^j)- We 
mainly use this property to obtain the results in the 
present study. 

In order to analyze the dynamical property of the Ising 
spin glass, let us suppose that the system evolves fol- 
lowing a stochastic dynamics as governed by the master 
equation. In the present study, we consider to change 
the inverse temperature. This case is found in a generic 
solver of the optimization problem as the simulated an- 
nealing [l6j]. We will formulate our theory for discrete 
time steps for simplicity, although the continuous case 
can be treated similarly. We change the coupling j3 from 
ft at t = to Pt at t — T in T steps of time evolution, 
(/3o, 01,"' iPt)- Correspondingly, the spin configuration 
changes as S t . The path probability for expressing the 
dynamical behavior of the system is then written by the 
product of the the transition rate from state St to state 
S t+ i following the master equation P t (St+i |S t , {ly}). 
The transition rate is also gauge-invariant, since it de- 
pends on the form of the Hamiltonian [l5|, [I?} • 



work is reduced to the ordinary work as 

5Y t (S t+l \{ Tij }) = /3if t+1 (S t+1 |{ nj -},)-/3if t (S t+1 |K-}) 
= ^W t (S t+1 |{^}). (5) 

For instance, changing the magnetic field as in Ref. [l2[ 
is the case. The performed (pseudo) work consists of 
collection of the discretized pseudo work 

T-l 

F({S t }, {Tij}; 0^T) = J2 SY t ({ nj }, S t+ i). (6) 

t=o 

Notice that the performed work depends on the specific 
configuration of {Tij}. 

We review the previous study for the performed work 
in spin glasses in short [IH, [l8[ . We applied the Jarzyn- 
ski equality to the special case for spin glasses in the 
previous study. The Jarzynski equality states that the 
expectation of the exponentiated work during nonequi- 
librium process is given by the difference of the free en- 
ergy (—j3F = log {tu})) between the initial and fi- 
nal conditions as [10|, LU| 

(e- Y ) ^ T =e- A °^ F \ (7) 

where A ^ T (pF) = p T F{fc;{nj}) ~ A)Wo;K-}). 
The brackets denote the nonequilibrium average over all 
realizations of the spin configurations in a nonequilibrium 
process starting from the equilibrium state defined as 

T-l 

(• ' ' >o-t = E II ^( s *+il s *> {Tij})P£ (So\{nj}), 
{S t } t=o 

(8) 

where P e ' q (S|{ry}) denotes the equilibrium distribution 
function 

<.(S|K}) = — 1 e -^ s l^-». (9) 

Z {Pu{Tij}) 



III. JARZYNSKI EQUALITY FOR SPIN 
GLASSES 

We analyze the work distribution function for spin 
glasses with a number of spins in the present study. For 
convenience, we fix several notations and review the pre- 
vious study before demonstrating the detailed analysis 
on our issue. 



A. Jarzynski equality 

We define the discretized pseudo work given by the 
difference of the inverse temperature as 

SY t (S t+1 \{n 3 }) = p t+1 H(S t+1 \{nj}) - p t H(S t+1 \{nj}). 

If we change the parameters in the Hamiltonian instead 
of the inverse temperature, then the discretized pseudo 



B. For spin glasses 

Since the work is given by the realization of the spin 
configurations at each time, the path probability can 
be regarded as the distribution function of the per- 
formed work with the specific configuration of {t^ } as 
P(y; {Tj}, —> T). However, in spin glasses, we are often 
interested in the averaged quantity over all realizations 
of {Tj}. In the previous study, the author and co-worker 
evaluated the averaged Jarzynski equality as 




where Nb is the number of bonds, and the square brack- 
ets denote the configurational average over all realiza- 
tions of {t^} defined as 

(11) 
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The above equality holds for the special initial condition 
that the nonequilibrium process starts from the Nishi- 
mori line f3 p — Pq HI} • Then the averaged free en- 
ergy on the right-hand side can be reduced to a triv- 
ial quantity. However the Jarzynski equality is for the 
expectation, which is the average over all realizations. 
We cannot obtain the detailed structure of the distribu- 
tion functions only from the expectation. In the present 
study, we thus revisit the problem on the performed work 
during nonequilibrium process in spin glasses by evaluat- 
ing the distribution function in a different way. That is 
the motivation of our study. 



IV. FLUCTUATION THEOREM FOR SPIN 
GLASSES 

A. Large deviation 

Throughout the present study, we assume the large de- 
viation property in the distribution function for the sys- 
tem with a large number of components. For instance, 
the distribution function P(y; {nj}, — > T) of the per- 
formed work for the specific configuration of {nj} takes 
an asymptotic form as, for a large N, 



P(y;{nj},0^T) 



^-NI(y-{r^},0^T) 



(12) 



where I(y; {nj}, — > T) is the rate function and always 
takes a non-negative value. At the most frequent realiza- 
tion of the performed work (thermodynamic work), the 
rate function vanishes. Here y stands for the scaled work 
defined as 



y = Y({S t },{ nj };0^T)/N. 



(13) 



In the thermodynamic system, the empirical average as 
the above scaled work can be evaluated by the zero point 
y* of the rate function and coincides with the expecta- 
tion, since 



J dyP(y;{ nj },0^T)y = y* 



(14) 



On the other hand, the rate function can characterize the 
fluctuation around the zero point. For the rate function, 
an important relation, the fluctuation theorem, holds. As 
detailed in Appendix the fluctuation theorem states 
the symmetry of the rate functions as 

I(y;{ Tij },0^T) 
= I(-y ] {r ij },T^0) 

-y- l(logZ(/3 T ;{r y }) - log Z(/3 ; {r y }))(15) 

This symmetry of the rate functions yields the well- 
known fluctuation theorem (Crook's fluctuation theo- 
rem) for the distribution functions of the performed work 
as 



P(y;{r l3 },0^T) 
P(-y;{ n ,},T^0) 



Rather than the Jarzynski equality, namely the expec- 
tation, the fluctuation theorem provides more detailed 
information on the distribution function. Therefore we 
consider to find the fluctuation theorem to spin glasses 
in the present study. 



B. Generating function 

The above fluctuation theorem holds for the specific 
configuration of {nj}, similarly to the Jarzynski equality. 
However, in spin glasses, we find sample-to-sample fluc- 
tuation in observations for different realizations of {nj}- 
We thus must evaluate the fluctuation around the most 
probable realization for the infinite-size system. By the 
analysis of the generating function of the distribution 
function of {nj} associated with the quantity we are 
interested in, we can evaluate such a sample-to-sample 
fluctuations by the rate function. Let us define the fol- 
lowing generating function of the performed work for spin 
glasses as 

n v (r; (3 p , -+ T) = 1 log [(erp(rNy))o^ T ]p p , (17) 



and of its inverse process, 

Q y (r; (5 p ,T^0) = ^ log [<exp(-riVy)) T ^ ]^ . (18) 

It is convenient to define the generating function for the 
specific configuration of {nj } as 

*v(r;{T tf },0->T) = ^ log(exp(rA^)) ^ T , (19) 
and for its inverse process 

* y (r;{ nj },T^ 0) = llog(exp(r7V(-y))) T ^ . (20) 
Notice that the definition of £l y (r;f3 p ,Q — > T) reads, by 

Eq. cm 



exp(iVn y (r;/? p ,0->T)) = [(exp(rNy))o^ T ] p 



(21) 



exp{A%-A ^ T (/3/))}. (16) 



For each realization of {nj}, the fluctuation theorem 
for the generating function holds as (See Appendix |A"| 

%(r; {nj}, T) = %(-(r + 1); {nj}, T -> 0) 
+ ^ (log Z{h- {nj}) - log Z(/3 ; {nj})) ■ (22) 

We thus obtain 

e W*,,(r;{Ty},0->.T) 
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Therefore we evaluate the exponentiated generating func- 
tion as 

Nn y (r:/3 P ,0^T) 



Rate function and fluctuation theorem 



s JV* s (r;{Ty},0-^T) 



Z(Pt] {Tjj}) jy<g B (-(r+l);{Tjj},T->-0) 



(24) 



The gauge transformation yields, as detailed in Appendix 

m 

2 N (2cosh(3 p ) NB e NQ y^ ^ 
= J2 ZfainMe'rW'WW 



E 

{r«} 



{Tij})Z((3Ti {Tjj}) N^ v (-(r+l);{T ii }.T^yQ) 



(25) 



We set 



h> = Po 



and then obtain 



2 w (2cosh/3 ) JV *e wn ^ ; ^ T > 

{T«} 

= ^ Z(/3 T ;{r ij })e JV *v(-(r+i) ; {r«},T^o)_ (26) 

On the other hand, let us evaluate the exponentiated 
generating function of the inverse process 



^N(l y (r:/3 P ,T^Q) 



(27) 



• f 

When f3 p — (3t, the gauge transformation yields 

2^(2 COSh M N Be Nn y {r;P T ,T^0) 

= X;^(Ar;{r«})e^»^>^°) 

= Z (fa {Ty})e"*»(-( r+1 M T «>'°"> T >. (28) 

{t«} 

Since the second line of Eq. (j2"6"]l is equal to the third line 
of Eq. (|28p except for the arguments of the generating 



function ty y , we reach 



2 cosh (3? 
2 cosh Bo 



Ms 



3 iVn w (r;/3T,T->0) 



(29) 



The relation (|29]l yields the symmetry of the rate func- 
tion of the performed work for spin glasses. We assume 
that the existence of the rate function for a large N as 



P(y;P P ,0^T) ~e 



-NJ(y;P p ,0-+T) 



(30) 



From the generating function we evaluate the rate func- 
tion through the Legendere transformation as 

J(y; B p , -> T) = sup {ry - Q y (r; B p , -> T)} , (31) 

r 

and that for its inverse process as 

J(y;B p ,T^0)=sup{ry-n y (r;B p ,T^0)}. (32) 

r 

Then the relation yields 

J(y;/3o,O^T) 
= Bup{n,-n»(r;A>,0->T)} 

r 

= sup^r + lJy-flvHr + lJjjSr.T-^O)} 

r 

' 2 cosh " 



2 cosh (3q 



where e? = Ng/N. Consequently, we obtain the fluctua- 
tion theorem for spin glasses as 



P(2/;/3 ,O^T) /2cosh/3 T 



P(y;/3 T ,T^0) V2cosh/3 



Nb 



^Ny 



(34) 



The fluctuation theorem for spin glasses immediately 
reads 



[(exp(-JVy)) . 



•TJ/3q 



2 cosh /3t 
2 cosh Bq 



(35) 



which reproduces the Jarzynski equality for spin glasses 
(|29| . We here use the fact that we can replace the inte- 
gration over all realizations of the performed work as the 
average over all configurations of {t^} and {St} as 



[(■ ' ' )o^r] Pp = J dyP(y; B p , ->■ T) x • • • . (36) 

By taking logarithm of the fluctuation theo- 
rem (|34p and average to make the form of the 
Kullback-Leibler (KL) divergence Dkl(Pa\Pb) = 
J <1xPa{x) log (Pa(x)/Pb(x)), we obtain two inequalities 
as 



, ' 2 cosh Bt " , 
[(y)o^T] Po +d\og[ JZ. 1 > 0. 



and 



[(v)t-+o]bt - dlog 



2 cosh Bq 



2 cosh Bt 
2 cosh Bo 



> 0. 



(37) 



(38) 
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Thus we obtain 

[{v)o^t] 0o + [{y) T ^o} PT > 0. (39) 

The equality holds, when two of the distribution func- 
tions is the same, 

P(y;P T ,T -+0) = P(y;P ,0^T), (40) 

since the left-hand side of Eq. (|3"9"|) can be evaluated by 

[(2/)o^t] (3o + [{y)T^o](3 T 

dy (P(y; p ; -> T) - P(-y; (3 T , T -> 0)) 
P(y;/? ,0^T) 



xl °g pr 7 J - ( 41 ) 

\P{-y,PT,T -> o)/ 

If we consider the quasi-static process, then (y)o->T = 
A(/3/)o-i-T following the second law of thermodynamics 
(i.e. P(i/ ;j Sp,0 -> T) = P(A(/3/);^,0 -> T)). There- 
fore, for the difference of the free energy of spin glasses, 
we expect the existence of the similar relation to the 
above fluctuation theorem. 



V. FLUCTUATION THEOREM FOR FREE 
ENERGY DIFFERENCE 

A. Generating function 

Let us consider the sample-to-sample fluctuations for 
the free energy difference A(/3/) by dealing with the rate 
function. We define the generating function for the free 
energy difference as 



f 



*A W )(r; &,<)-> T) = - log [exp(riVAo^r (/?/))] ^ 



N 

The gauge transformation gives 

e iV*AC W )(r;&,,0->-T) 

Z(P P ;{n 3 }) (Z{(i T -{T l3 }) 



E 



{rtj} 

We set P P = Po and obtain 

e A r *AO/)(^;/3o,O^T) 



2^(2 cosh p p )"B \Z(Pa;{nj}) 



(42) 



(43) 



E 



2^(2 cosh (3 ) n b \Z((3 ;{ nj }) 



r-l 



(44) 



On the other hand the condition P p = Pt leads to 

- 2. ^ 2^(2 cosh /3 T )^ U(A,;{T })j - 1 ' 

Therefore we find a relation of the generating function as 
*A(/3/)(r- + l;A>,0^T) 

' 2 cosh (St " 



*A03/)(r;/3T,0^T) + dlog^ 



2 cosh /3q 



.(46) 



B. Rate function and fluctuation theorem 

We assume that the large-deviation property holds for 
the free energy of the large system N — > oo as 

P(Ao_> r (0/) ; /3 p ) ~ exp {-NK(A^ T (Pf)-p p )) ■ (47) 

We then obtain the rate function of the free energy dif- 
ference through the Legendre transformation as 

K(A ^ T (Pf);p p ) 

= sup {rA(Pf) - *A0S/)(r; Pp, T)} . (48) 

By use of Eq. (|46j) . we immediately find 
K(Ao^ T (Pf);Po) 



K(Ao^ T (Pf);j3 T ) - ACS/) - dlog 



2 cosh /3-r 
2 cosh /3q 



Therefore we find 



P(A ^r(/3/);ft)) = / 2 cosh /3 T ^ ^ A(/3/) 
P(A ^ T {pf);p T ) \2 cosh A). 



(49) 



(50) 



Due to the definition of the free energy difference, 
P(A ^ T (/3/);/3p) = P(-A T -*>03/);/9 P ), and we thus 
obtain the relation in the same form as the fluctuation- 
theorem, 



P(A Q ^ T (Pf);P ) = / 2cosh/? T 
P(-A T ^ (Pf);p T ) ^ 2 cosh A, 



N B 



3 JVA(/3/) 



(51) 



We also establish the Jarzynski-type equality as, by in- 
tegrating the exponentiated free energy difference, 



[cM-NA ^ T (pf))} Po = 



2 cosh Pt 
2 cosh Pa 



N B 



(52) 



We here use the fact that we can regard the distribution 
function of {t^} as that of the free energy difference as 



dxP(x : P p ) x 



(53) 



By making the form of the KL divergence, we obtain the 
following inequalities as 

[Ao^T^U+dlogfl^Ho. (54) 



and 



2 cosh po 



[A^o(/3/)W-dlog('^^l : <>. 



(55) 



2 cosh Po 
We thus obtain 

[A T ^{Pf)] h + [A^ T {Pf)] Pa > 0. (56) 
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The deviation from zero can be written as 



[At^o(/3/)W + [Ao^tW)] Po 



P{-x\Pt) 
P(x;/3 ) 



(57) 

The equality holds when P(A _> T (/?/); Pa) = 
P(— At->o(/3/);/3t)- Since two of the distribution 
functions P(A ^ T (/3/); p ) and P(-A T -*>(Af);#r) do 
not coincide with each other in general, the equality in 
Eq. (|56p is not expected to hold. Nor is the equality in 
Eq (|39|) . The magnitude of the violation of the equality 
can be evaluated by the quantity related to the KL 
divergence. 

As above considered the gauge symmetry leads to an- 
other type of the fluctuation theorem not only for the 
performed work but also the free energy differences for 
different configurations of {"%■}. In this sense the ob- 
tained relations (|3"4")l and ([5~Tj) are different from the or- 
dinary fluctuation theorem. Our results are related to 
the sample-to-sample fluctuations of the different realiza- 
tions of {rij}. The sample to sample fluctuation yields 
relevant effect even in equilibrium. In theoretical studies 
in spin glasses, we usually employ the replica method to 
evaluate the equilibrium property. In the next section 
we demonstrate to evaluate the equilibrium property for 
spin glasses with gauge symmetry from a perspective of 
the rate function without the replica method. As a re- 
sult, we find a different way to understand the peculiar 
behavior in spin glasses. 



VI. FLUCTUATION THEOREM FOR FREE 
ENERGY 

A. free energy statistics 

We again assume that the large-deviation property for 
free energy in a large- N system 

P(f;p p ,p)~eiq>(-NL(f;P p ,p)). (58) 

Here we regard the distribution function of {r^ } as that 
of the free energy. We define the generating function of 
the free energy as 

*f(r;Pp,P) - ^log[exp(rJV(-/3/))]^ . (59) 
The exponentiated generating function is 

eF*' ( ™ = [Z r (P;{Tn}))] fir . (60) 
The analysis by the gauge transformation lead us to 
*f(r;P P ,P) 



= -log 2 - dlog(2cosh(/3p)) 



— log ^Z(/3 p ;{r y })^(/3;{r y })j .(61) 



On the Nishimori line (3 P = /3, we find 

V f (r;P,P) = -log 2 - dlog(2cosh/3) 



+llog ^E^ r+1 (/3;{ry})j . (62) 



We obtain the following similar quantity in the symmet- 
ric distribution (P p = 0) 



*/(r; 0, p) = -dlog2 + 1 log [ >>' ,■ } i 



B. Fluctuation theorem 

We find a relation from the above generating functions 

0, P) + log 2 + d log (cosh/3) = */(r + l;0,/3), (63) 

which is essentially the same as the calculation in Ref. 
[l9j . The above relation enables us to analyze the critical 
behavior of the spin glasses in the symmetry distribution 
through the free energy on the Nishimori line as shown 
in Ref. [20j. However, in a modern point of view, this re- 
lation can be regarded as the fluctuation theorem for free 
energy. Indeed we find the symmetry of the rate function 
of the free energy through the Legendre transformation 
as 

L(f; p, /3) = L(/; 0, /3) + /3/ + log 2 + d log (cosh p) , (64) 
where we defined the rate function as 

L(f; p p , P) = sup {r(-Pf) - * y (r; p p , P)} . (65) 

r 

Thus we obtain the fluctuation-theorem type equality for 
the free energies on the Nishimori line and in the sym- 
metric distribution as 



-AT/3/ 



P{f;P,P) = 

P(/;0,/3) 2" (cosh 0) 



N B 



(66) 



By taking the logarithm and average to make the form 
of the KL divergence, we obtain two of inequalities as 



and 



■Pf*(P,P) > log 2 + dlog (cosh/3) (67) 



Pf*(0,P) <log2 + dlog(cosh/3), (68) 



where we defined the free energy in the thermodynamic 
limit as 



(69) 



f*(P p ,P)= / dfP(f;p p ,P)f 
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FIG. 1: Phase diagrams of spin glasses. (A) The typical 
phase diagram of several mean-field models and that of Mat- 
tis model (then T c — Tsg)- (B) typical instance of the finite- 
dimensional ±J Ising model. The vertical axis denotes the 
temperature and the horizontal one expresses the density of 
the quenched randomness. The dashed line depicts the Nishi- 
mori line (/3 P = /?) . The solid lines separate the representative 
phases: the ferromagnetic (Ferro), paramagnetic (Para) and 
spin glass (SG) ones. In (B), below the dotted line, the Grif- 
fiths paramagnetic phase is expected to be laid. 



In addition, since f a (0,/3) is a trivial function, 
the non-analytical point of the free energy /* 
is identified as that of the KL divergence. If 
two of the KL divergences D KL (P(f; (3, /3)|P(/;0, (3)) 
and D KL (P(/;0,/3)|P(/;£,/3)) have the non- analytical 
points at the same temperature, the phase boundary can 
be parallel to /3 p -axis from the Nishimori line to the re- 
gion in the symmetric distribution. Notice that not nec- 
essarily it means two of the KL divergence coincide with 
each other. The case is expected to be the Griffiths sin- 
gularity, which is considered to be located at the same 
temperature T c for any /3 p as the ferromagnetic transi- 
tion point without the quenched randomness [23l - |25j as 
in the case of (B) in Fig. [TJ In this sense, the Griffiths 
singularity might be specified as the appearance of the 
simultaneous non-analytical point of two symmetric KL 
divergences of the free-energy distribution functions. It 
would push up the understanding of the Griffiths sin- 
gularity with the aid of the information geometry [26j 
through the above consideration. 



VII. CONCLUSION 



C. Phase diagrams in spin glasses 

The common quantity on the right-hand sides of Eqs. 
(|67]l and ([68| is equal to the annealed free energy 
given in the symmetric distribution as — /3/ Q (0,/3) = 
log[Z(/3; {Tij})]p p= o/N. Thus the inequalities ([67]) and 
([68]) read 

/*(/3,/3)</ a (0,/3) (70) 

and 

/*(0,/3) >fa{0,P). (71) 

From the fluctuation-theorem type relation (|66[) . the vi- 
olation of the equality relates to the KL divergence as 

/*C8, 0) - /a(0, (3) = -~Z> K l(P(/; 0, /3)|P(/; 0, /9)) 

(72) 

and 

r(0,/3)-/ a (0,/3) = ^D KL (P(f;0 7 (3)\P(f;(3,(3)). (73) 

When f*(0,/3) = f a (0,P), we immedi- 
ately find D KL (P(f;p,p)\P(f;0,P)) 
£>kl(P(/;0,£)|P(/;/3,/3)) = 0. Thus we conclude 
that /*(j8,j8) = /*(Q,j8). This relation ensures that the 
critical point on the Nishimori line is located at the 
same temperature in the symmetry distribution, when 
/*(0, (3) = f a (0,l3) as often seen in the paramagnetic 
solutions for the mean-field spin glass models and the 
free energy of the Mattis model [2ll [22j as in the case 
of (A) in Fig. [T] These models show the parallel phase 
boundary to /3 p -axis from the Nishimori line to the 
region in the symmetric distribution. 



We analyzed the distribution function of the performed 
work for the spin glass with the gauge symmetry. The 
gauge symmetry revealed the existence of the symme- 
try in the rate function of the performed work as well 
as the free energy depending on each realization of the 
quenched randomness. As a result we obtained several 
relations in the same form as the fluctuation theorem. 
In order to analyze spin glass system, we usually use the 
replica method, which deals with the highly-correlated 
multiple system of the original model. Although a part 
of our results overlapped with the known properties via 
the replica method and gauge transformation as we re- 
covered, our analyses was directly performed on the dis- 
tribution function without replica method and related 
the averaged quantities as the performed work and free 
energy to the KL divergence. In this sense we believe 
that our analyses should be valuable to provide a dif- 
ferent perspective to understand the nonequilibrium and 
critical behaviors for spin glasses. 
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Appendix A: Crook's fluctuation theorem 

We here demonstrate the Crooks's fluctuation theorem 
[l[ of the performed work employing derivation by use of 
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the large-deviation principle as done for the long-time 
behaviors by the Lebowitz and Spohn Q ■ Instead of the 
long time observation (T 3> 1), we consider the large 
number of components (N ^> 1) in the present study. 

We assume that the detailed balance condition is sat- 
isfied as 



P t (S t+1 |S t , {-%■}) _ Pt q .(St+i\{nj}) 



P t (S t \S t+1 ,{ Tij }) P* q .(S t \{r tJ }) 
Then the product over all steps can be expressed by 

jj i^St+llSt.-fo}) = T fr p^.(s t+1 \{ nj }) 

11 p t (s t |s t+ i,K-}) 11 



(Al) 



t=o 



n* q .(s t |K-}) 



t=0 c <i 
3 - Ef^^CSt+i.StKry}) 



(A2) 



where we defined the discretized pseudo heat as 
<DQ(S t+ i,S t |{r ti }) = ^(St+ilK-}) - p t H(S t \{Tij}). 
The (pseudo) heat is given by 



T-l 



X({S t }, ^ T) = ]T SX t (S t+1 , S t |{r y }). (A3) 

t=o 

Then we confirm the first law of thermodynamics as 

Y({S t }, {nj}; — > T) + X({S*}, {nj}; -> T) 

- P T H(S T \{ nj }) - A)iT(So|{Ty}). (A4) 

Therefore we reach a relation between the original pro- 
cess and its inverse one starting from the equilibrium 
states as 



r-i 



HP t (S t+1 \S t ,{ nj })e 



Y({S t },{T is };0->T) 



4=0 
T-l 



= J] P t (St|S t+1 ,{^})e-^ H ( s -l^«+^ H ( s °l^». 
t=o 

(A5) 

As a result, we find the following relation as 



T-l 



n^(St+l|St,{^})i^.(So|{Ty}) 
t=0 

_ Z(/3t;K-}) 
^o;H-}) 

T-l 

x H P t (S t |S t+1 ,{r y })P T q .(S T |{r y })e 



(A6) 

This relation yields Eq. (j22]t . 

The rate function is given by the Legendre transfor- 
mation of the generating function as 

I(y; { nj }, -> T) = sup {ry - *„(r; {ry}, -> T)} . 

r 

(A7) 



On the other hand, the rate function for the inverse pro- 
cess is also defined as 

I(y; { Tij },T -> 0) = sup {ry - * y (r; {t w },T -> 0)} . 

r 

(A8) 

By use of the relation (f22| , we find 

/(y;K},0->r) 

= sup {(r + l)y - * y (-(r + 1); { nj },T 0)} 

r 

-y - ^(log'Z(/? T ; { TiJ }) - logZ(/3 ; {ry})) 

= /(-y;{7 ii },T->0) 

-y- ^(logZC8r;{r tf }) - log Z(/3 ; K })).(A9) 

This symmetry of the rate functions yields the well- 
known fluctuation theorem for the distribution functions 
of the performed work as 



P{y-{r l3 },Q^T) 

P(-y;{ T y},T->0) 



= exp{JVfo-Ao- T G8/))} 



(A10) 

This is the Crook's fluctuation theorem pj. Notice that 
this derivation of the fluctuation theorem through the 
analysis of the generating function is trivial but has not 
been seen as far as the best of our knowledge. 



Appendix B: Gauge transformation 

We demonstrate the manipulation of the gauge trans- 
formation to obtain Eq. (|25|) from Eq. ([24)) . The quan- 
tity in the second line of Eq. (fMf can be written as 



3 JV* tf (r;{Ty},0-vT) 



E 

{^} 



n 



(m 1 



(2cosh/3 p )^ 



a JV* y (r;{T„},0-vT) 



(Bl) 



The gauge transformation yields 



,N* v (r;{T i:i },0^T) 



E 

{r«} 



(2cosh/3 p ) 



N F . 



,N* y (r;{T i:i },0->T) 



(B2) 



Thus we sum over all possible configurations of {ai} and 
obtain 



2 N \ e N^ y (r:{r t] },Q^T) 



P 



^(2cosh/3 p )^ e ■ (Bdj 
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Similarly, we can evaluate the quantity in the third line The summation over all possible configurations of {<Ji\ 
of Eq. flip as yields 



Ltf* B (r;{T„},0- 



■T) 



Pp 



E Z (Pt;{Tij}) Z (/3 p ; {Tjj}) Af^„(-(r+l);{T„}-T^0) 
Z{fa; { TiJ }) (2 cosh p p ) N * 



^*»(r;{r«. 



},T->0) 



ft. 



(B4) 



, . (2cosh/3 p )^ 



(B7) 



Therefore we reproduce Eq. ()25|) . 

In addition, the quantity in the second line of Eq. (j2"7| 

is 



r e JV* y (r;{T< 



The same analysis can be applied to the third line of Eq. 
([27| as 



j},T-yQ) 



E 

fry} 



n fa ->^ 

(2cosh/3 p )^ 



,JV* v (r;{r«},T^0) 



(B5) 



|^ e iV* B (r;{T 4; 



},T->-0) 



The gauge transformation yields 

^ e JV* a (r;{r«},r-)-0)" 



^(Ab { T »j}) Z (/3 p ; {Tjj}) r N1> y (-(r+l);Uii}.0^T) 

(B8) 



E ■ z(^ T ;{r ij })(2cosh^. ? 

L T ^j J 



n 



E ( 2c osh/3 p ) 



a iV* y (r;{ry},T-).0) 



(B6) When /3 P = /3 T , we find Eq. 
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